Beyond the Score

Reporting Equity and Justice in Educational Accountability

Damian Betebenner

June 24th, 2025

Uh oh!

I was clueless!

  • Going back several years, I’d hear colleagues frequently using the term “equity” to characterize the work they were doing.
  • It seemed like a good thing. It’s not like I support inequity.
  • But in the end, I really didn’t know what it meant.
  • I’d heard about DEI programs and interventions that characterize themselves as equity-focused.
  • But those are inputs and I’m an outputs guy.
  • I was curious: How would one characterize/recognize and ultimately quantify equity in terms of the outputs we all know and love - state summative assessment results?
  • So what did I do? This was pre-ChatGPT so I started Googling.

The Famous Infographic

This Gave Me Some Ideas – Here’s What I Saw

  • I saw the fence as a metaphor for the “proficiency” requirements and the individuals seeing over the fence as demonstrating proficiency.
  • I saw disparate proficiency outcomes represented by some individuals seeing over the fence and others not.
  • I saw Equality, where “input” boxes are of equal height, not eliminating proficiency disparities.
  • I saw Equity, where “input” boxes adjust in height to the need of the individual, leading to the elimination of proficiency disparities.
  • I felt like I was starting to get it.

Then I Read This!

Sippin the EquiTEA made some good points!

  • Making the children different heights implicitly blames the student for not being able to see over the fence.
  • The graphic doesn’t address the uneven playing field associated with the systemic inequities that exist in our society.
  • So I wanted to see if I could come up with a graphic that merges these ideas together.

Here’s What I Came Up With

I’m Not Saying

  • This is perfect or better than other illustrations.
  • This is original or the best way to think about these ideas.
  • It did, however, help me get some traction on how ideas like equity and justice could be quantified in terms of the outputs we all know and love - state summative assessment results.
  • That’s what I’m going to run with for the rest of this presentation.

Equity

Making Sense of Equity

  • The boxes represent the equity “contribution”.
  • The contribution includes, for example, supplemental learning interventions and supports to help students who are behind “catch up”.
  • The height of the boxes represents the magnitude of the contribution or, conversely, the magnitude of the inequity (academic headwinds) to overcome.
  • These contributions would manifest themselves as higher rates of growth (academic tailwinds) for those students.
  • The critical insight here is that student growth is the basis for investigating equity.

Student Growth as an Equity Indicator

  • To investigate equity vis-à-vis student growth, we need a measure of student growth that allows different groups to demonstrate different rates of growth.
  • If one only wants to make between group equity comparisons within a given year, then norm-referenced SGPs are sufficient.
  • If one want to investigate equity over time, for example, at the state level, then baseline referenced SGPs are required.

Visualization - Growth as an Equity Indicator (Baseline Scenario)

Visualization - Growth as an Equity Indicator (Baseline Scenario)

  • The mosaic plot depicts percentages of students demonstrating different levels of growth based upon the growth needed by the student.
  • The widths of the columns represent the current percentage of students in each category (in this figure 50%, 25% and 25% have growth goals of > 1 Year’s Growth, 1 Year’s Growth, and < 1 Year’s Growth, respectively.
  • The heights of the rectangles represent the percentage of students in each row category demonstrating that amount of growth (uniform distribution of growth by prior achievement).
  • With cohort-referenced SGPs or baseline-referenced SGPs in the baseline year, SGPs will be uniformly distributed.
    • For cohort-referenced SGPs, SGPs for subgroups (e.g., a district) needn’t be uniformly distributed within year.
    • For baseline-referenced SGPs, SGPs for the entire state or for subgroups needn’t be uniformly distributed following the baseline year.

Visualization - Growth as an Equity Indicator (Good Scenario)

Visualization - Growth as an Equity Indicator (Good Scenario)

  • This figure shows a good scenario with regard to equity.
  • The growth associated with the students needing the equity boost to catch-up shows a preponderance of students demonstration a least a year’s worth of learning.
  • Decreasing the percentage of students demonstrating less than a year’s worth of learning is key.

Justice

Making Sense of Justice

  • The downward slope of the ground represents the myriad of systemic factors (many outside of the control of the education system) that impact student outcomes.
  • Equity, the boxes, represents what needs to be done to overcome these systemic disparities.
  • Justice entails the amelioration of the systemic factors that impact student outcomes.

Systemic Factors

  • There’s a corollary to the way justice is characterized to some different assumptions that current growth models make.
  • Often, we use demographic variables as proxies to account for systemic factors that impact student outcomes.
  • Many value-added models incorporate individual and group level demographic characteristics to account for the impact of systemic factors on student outcomes.
  • I would argue, in line with Peter Drucker, that if our interest is in ameliorating systemic factors that impact student outcomes, then we need to measure them directly.
  • Bill Sanders, when discussing the use of demographics in value-added modeling, feared that their incorporation was “sweeping the problem under the rug”.

Next Steps

The Future of Equity and Justice in Educational Accountability

  • Begin running growth based equity analyses and visualizations for interested state clients.
  • Move beyond demographic proxy variables and incorporate more direct measures of systemic factors that impact student outcomes.
  • Consider how such indicators can be used to inform policy and practice and potentially as part of accountability systems.